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Abstract 

Evolutionary algorithms (EAs) are heuristic algorithms inspired by natural evolution. They are often 
used to obtain satisficing solutions in practice. In this paper, we investigate a largely underexplored 
issue: the approximation performance of EAs in terms of how close the solution obtained is to an 
optimal solution. We study an EA framework named simple EA with isolated population (SEIP) that 
can be implemented as a single- or multi-objective EA. We analyze the approximation performance 
of SEIP using the partial ratio, which characterizes the approximation ratio that can be guaranteed. 
Specifically, we analyze SEIP using a set cover problem that is NP-hard. We find that in a simple 
configuration, SEIP efficiently achieves an i?„-approximation ratio, the asymptotic lower bound, for the 
unbounded set cover problem. We also find that SEIP efficiently achieves an (Hk — ^^-approximation 
ratio, the currently best-achievable result, for the fc-set cover problem. Moreover, for an instance class of 
the fc-set cover problem, we disclose how SEIP, using either one-bit or bit-wise mutation, can overcome 
the difficulty that limits the greedy algorithm. 

Key words: Evolutionary algorithms, approximation algorithm, approximation ratio, fc-set cover, 
time complexity analysis 



1. Introduction 



Evolutionary algorithms (EAs) |3[ have been successfully applied to many fields and can achieve ex- 
traordinary performance in addressing some real-world hard problems, particularly NP-hard problems 
[it], Qj- To gam an understanding of the behavior of EAs, many theoretical studies have focused 



on the running time required to achieve exact optimal solutions 
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In practice, EAs are 
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most commonly used to obtain satisficing solutions, yet theoretical studies of the approximation ability 
of EAs have only emerged recently. 



He and Yao [15[ first studied conditions under which the wide-gap far distance and the narrow-gap long 
distance problems are hard to approximate using EAs. Giel and Wegener |l2[ investigated a (1+1)-EA 
for a maximum matching problem and found that the time taken time to find exact optimal solutions 
is exponential but is only 0(n 2 ^ 1/,e ^) for (1 + e)-approximate solutions, which demonstrates the value 
of EAs as approximation algorithms. 

Subsequently, further results on the approximation ability of EAs were reported. For the (1+1)-EA, 
the simplest type of EA, two classes of results have been obtained. On one hand, it was found that 
the (1+1)-EA has an arbitrarily poor approximation ratio for the minimum vertex cover problem and 
thus also for the minimum set cover problem [ill [2^]. On the other hand, it was also found that 
(1+1)-EA provides a polynomial-time randomized approximation scheme for a subclass of the partition 
problem [32(. Furthermore, for some subclasses of the minimum vertex cover problem for which the 
(1+1)-EA gets stuck, a multiple restart strategy allows the EA to recover an optimal solution with 
high probability [281 ] . Another result for the (1+1)-EA is that it improves a 2-approximation algorithm 
to a (2 — 2/n)-approximation on the minimum vertex cover problem jlQ| . This implies that it might 
sometimes be useful as a post-optimizer. 

Recent advances in multi-objective (usually bi-objective) EAs have shed light on the power of EAs as 
approximation optimizers. For a single-objective problem, multi- objective reformulation introduces an 
auxiliary objective function for which a multi-objective EA is used as the optimizer. Scharnow et al. 
[30) 1 first suggested that multi-objective reformulation could be superior to use of a single-objective EA. 
This was confirmed for various problems 24 , IllL Hi by showing that while a single-objective EA 



could get stuck, multi-objective reformulation helps to solve the problems efficiently. 

Regarding approximations Jit has been shown that multi-objective EAs are effective for some NP-hard 
problems. Friedrich et al. IjJ proved that a multi-objective EA achieves a (lnn)-approximation ratio 
for the minimum set cover problem, and reaches the asymptotic lower bound in polynomial time. 



Neumann and Reichel 



showed that multi-objective EAs achieve a fc-approximation ratio for the 



minimum multicuts problem in polynomial time. 

In the present study, we investigate the approximation ability of EAs by introducing a framework 
called simple evolutionary algorithm with isolated population (SEIP), which uses an isolation function 
to manage competition among solutions. By specifying the isolation function, SEIP can be implemented 
as a single- or multi-objective EA. Multi-objective EAs previously analyzed [3, [ill 13 can be viewed 
as special cases of SEIP in term of the solutions maintained in the population. By analyzing the SEIP 
framework, we obtain a general characterization of EAs that guarantee approximation quality. 
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We then study the minimum set cover problem (MSCP), which is an NP-hard problem [9(. We prove 
that for the unbounded MSCP, a simple configuration of SEIP efficiently obtains an iJ^-approximation 
ratio (where Hk is the harmonic number of the cardinality of the largest set), the asymptotic lower 
bound [9(. For the minimum fc-set cover problem, this approach efficiently yields an (H^ — ^pr)- 
approximation ratio, the currently best-achievable quality (13j |. Moreover, for a subclass of the minimum 
£;-set cover problem, we demonstrate how SEIP, with either one-bit or bit- wise mutation, can overcome 
the difficulty that limits the greedy algorithm. 

The remainder of the paper is organized as follows. After introducing preliminaries in Section 2, 
we describe SEIP in Section 3 and characterize its behavior for approximation in Section 4. We then 
analyze the approximation ratio achieved by SEIP for the MSCP in Section 5. In Section 6 we conclude 
the paper with a discussion of the advantages and limitations of the SEIP framework. 

2. Preliminaries 

We use bold small letters such as w, x, y, z to represent vectors. We denote [m] as the set {1,2,..., m] 
and 2 s as the power set of S, which consists of all subsets of S. We denote H n = Y^i=i 1 f° r the n ^ D - 
harmonic number. Note that H n ~ Inn since Inn < H n < Inn + 1. 

In this paper, we consider minimization problems as follows. 
Definition 1 (Minimization problem) 

Given an evaluation function f and a set of feasibility constraints C, find a solution x G {0, 1}™ that 
minimizes f(x) while satisfying constraints inC. A problem instance can be specified by its parameters 
(nJ,C). 

In the definition of the minimization problem, solutions are represented in binary vectors. When the aim 
of a minimization problem is to find a subset from a universal set, we can equivalently use a binary vector 
to represent a subset, where each element of the vector indicates the membership of a corresponding 
element of the universe set. For example, given a universal set U = {1, 2, 3, 4, 5}, its subset S = {I, 3, 5} 
can be represented by a binary vector v = (1,0,1,0,1), and we define U(v) = S. Considering the 
equivalence between sets and binary vectors, we apply set operators (| • |, n, U, — , C, g) to binary vectors 
when there is no confusion. For example, |(1,0, 1,0, 1)| = 3, (1,0, 1,0, 1) n (0,0,0, 1,1) = (0,0,0,0, 1), 
and (1, 0, 1, 0, 1) - (0, 0, 0, 1, 1) = (1, 0, 1, 0, 0). We denote x $ = (0, . . . , 0) as the vector corresponding 
to the empty set. 

We investigate the minimum set cover problem (MSCP), which is an NP-hard problem. 
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Definition 2 (Minimum set cover problem (MSCP)) 

Given a set of n elements U — [n] and a collection C — {Si, S%, . . . , S m } of m nonempty subsets of U, 
where each Si is associated with a positive cost w(Si), find a subset X* C C such that J2sex* W (S) 

is minimized with respect to U S = U . 

sex* 

Using binary vector representation, we denote an instance of the MSCP by its parameters (n, w, C, U), 
where \C\ = m and w is the cost vector. The MSCP involves finding a vector x OPT , which is equivalent 
to its set representation X* , by solving a constrained optimization problem 



OPT 

x = arg mm w ■ x 
xe{o.i} m 

s.t. (j s = u, 

sec(x) 

where w-x is the inner product between vectors w and x, and C(x) denotes a set consisting of elements 
in the collection C that are indicated by binary vector x. 

Definition 3 (Minimum fc-set cover problem) 

An MSCP (n, w, C, U) is a k-set cover problem if, for some constant k, it holds that \S\ < k for all 
S € C, denoted as (n, w, C, U, k). 

A solution is called feasible if it satisfies all the constraints in C; otherwise, it is called an infeasible 
solution, or a partial solution in this paper. Here, we assume that the evaluation function / is defined 
on the full input space, that is, it evaluates all possible solutions regardless of their feasibility. For 
example, if we evaluate a solution x of the MSCP using w x, this evaluation function can be calculated 
for any solution. 

Given a minimization problem, we denote x OPT as an optimal solution of the problem, and OPT = 
f(x OPT ). For a feasible solution x, we regard the ratio Qpj. as its approximation ratio. If the 
approximation ratio of a feasible solution is upper-bounded by some value r, that is, 

/(as) 
1 < < r. 

- q PT - 

the solution is called an r- approximate solution. An algorithm that guarantees to find an r-approximate 
solution for an arbitrary problem instance in polynomial time is an r- approximation algorithm. 

The greedy algorithm [5] described here as Algorithm[T]is the most well-known approximation algorithm 
for the MSCP. This greedy algorithm consists of a sequence of steps. The cost of a candidate set is 
defined as its weight divided by the number of its elements that have not been covered yet (i.e., the 
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quantity r$ in line 3). The algorithm then picks the candidate set with the smallest cost for the 
solution (line 4) and marks the newly covered elements (line 6). This simple algorithmyields an H n - 
approximation ratio or, more exactly, Hk, where fc is the cardinality of the largest set [5j. The key to 
the proof of the approximation ratio is the definition of the price of elements, as in line 5. The price 
of an element equals the cost of the set that first covers it; therefore, the total price of all elements 
equals the total cost of the solution. Furthermore, it should be noted that when an element is covered 
by a set with the lowest cost, it would also be covered by one set of an optimal solution but with a 
higher cost. Therefore the price of the element is upper-bounded by the optimal cost and hence the 
approximation ratio is upper-bounded. For a detailed proof please refer to Chvatal [5[ . 
Algorithm 1 (Greedy algorithm Q|) 

Given a minimum k-set cover problem (n,w,C,U,k), the greedy algorithm consists of the following steps: 



1: X ^ 0;i? ^ 
2: while R^U do 

3: VSeC:\S-R\>O,letr s ^-0 

4: S arg min rs 

s 

5: let priceie) <— r§ for all e £ S — R 
6: let R 4- R U S, and X <- X U {S}. 
7: end while 
8: return X 



Several studies have shown that the approximation ratio of the MSCP is lower-bounded by O(lnn) 

i SllflQl- Therefore, the greedy algorithm achieves the asymptotic 



unless P = NP that is unlikely 
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lower bound of the approximation ratio for the MSCP. 

Although the iJ n -approximation ratio is asymptotically tight for the unbounded MSCP, a better ap- 
proximation ratio can be achieved for the minimum fe-set cover problem, where A: is a constant. It 
has been proved that for the unweighted minimum fc-set cover problem, an (Hk — ^-a ppr oximation 



ratio can be achieved |8( and, if k > 4, an improved ratio (Hk — can be achieved [20|. For the 



weighted minimum fc-set cover problem, a greedy-algorithm-with- withdrawals (GAWW) was presented 
fc— 1 

8fc u 



and achieved an Hk — 4rr -approximation ratio |13| 



The GAWW algorithm presented here as Algorithm [2] is a modification of the greedy algorithm. In 
every iteration, the algorithm chooses between taking a greedy step, which is the same as in the greedy 
algorithm, and a withdrawal step, which replaces a set in the current solution with at most fc candidate 
sets. It evaluates the cost of candidate sets as in the greedy algorithm, and also evaluates the benefit 
of the withdrawal (calculated in lines 4 and 5). When the benefit of the withdrawal step is not large 
enough according to the criterion in line 6, the algorithm takes the greedy step, and otherwise takes 
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the withdrawal step. To prove the approximation ratio, the price of elements is defined similarly in 
line 7 for the greedy step and line 10 for the withdrawal step, which is used later in the proofs for this 
paper. 



Algorithm 2 (GAWW [13J) 



Given a, minimum k-set cover problem (n, w,C,U,k), GAWW consists of the following steps. 
1: X <- 0; R <- 0; a k <- 1 - £ 
2: whiJe R^U do 
3: VS G C : [S - R\ > 0, Jet r s 4- 

4: VS e X, Q C C : |Q| < fc A | U s , eQ 5' - i?| > 0, Jet r (s , Q) f^fff/T^f 

5: <§ <s— argminr^, and (£?, Q) «— argmin r^s,Q) {choose the minimal number of sets in cases of ties} 

S ' (S,Q):\Q\<k 

6: if rg ■ ctt < r,g q-. then {greedy step} 

7: let price(e) <— r§ for all e G S — R 

8: letR^RuS, and X <r- X U {§}. 

9: else {i.e., r,§ q) < r s ' ak ' withdrawal step} 
10: let price(e) «— »"(s,q) f° r a ^ e £ ^sgqS — R 
11: let R <r- UssqSIIR, and X <- X UQ - {S} 
12: end if 
13: end while 
14: return X 



The (1+1)-EA is the simplest EA implementation, as described in Algorithm [3] Starting from a 
solution generated uniformly at random, the (1+1)-EA repeatedly generates a new solution from the 
current one using a mutation operator, and the current solution is replaced if the new solution has 
better (or equal) fitness. 
Algorithm 3 ((1+1)-EA) 

Given a minimization problem (n,f,C), each solution is encoded as a binary vector of length m and the 
(1+1 )-EA-minimizing f consists of the following steps. 

1: x <s— a solution generated uniformly at random 

2: whiJe stop 7^ true do 

3: Mtztate x to generate x' 

4: if x is feasible and f(x') < f(x) then 

5: x <— x' 

6: end if 

7: end while 

8: return x 



Two mutation operators are commonly used to implement the "mutate" step in line 3: 
One-bit mutation: Flip one randomly selected bit position of x to generate x'. 



6 



Bit-wise mutation: Flip each bit of x with probability — to generate x'. 



It has been shown that the (1+1)-EA has an arbitrarily poor approximation ratio for the MSCP 
Laumanns et al. [19| used a multi-objective reformulation with a multi-objective EA named SEMO to 
achieve a (lnn)-approximation ratio. The SEMO algorithm is described in Algorithm |4l where two 
objectives are presented as (/i,/2)- To apply SEMO to the MSCP, let /i evaluate the cost of the 
solution and /2 evaluate the number of uncovered elements. Thus, SEMO minimizes the cost and the 
number of uncovered elements of solutions simultaneously. A notable difference between SEMO and 
(1+1)-EA is that SEMO uses a non-dominance relationship, implemented using the dominate function 
in SEMO. The population of SEMO maintains non-dominant solutions, that is, no solution is superior 
to another for both of the two objectives. 
Algorithm 4 (SEMO Q) 

Given a two-objective minimization problem (n, (/i, /a)), each solution is encoded as a binary vector of length 
m. SEMO minimization of /i) consists of the following steps. 

1: P^{x 9 = (0,0,...,0)} 
2: while stop 7^ true do 
3: Choose x £ P uniformly at random 
4: Mutate x to generate x' 
5: ifVa; £ P : dominate(a;, x') = false then 
6: Q {x € P I dominate(cc', x) = true} 

7: P ^ PU{x'} -Q 

8: end if 
9: end while 
10: return P 

where the dominate function of two solutions is defined such that dominate(a;, y) is true if any one of the 
following three rules is satisfied: 

1) fi(x) < fi(y) andf 2 (x) <= f 2 (y) 

2) fi(x) <= My) and f 2 (x) < f 2 (y) 

3) fi(x) = fi(y) and f 2 (x) = f 2 (y) and \x\ < \y\ 
and dominate (x, y) is false otherwise. 



3. SEIP 

The SEIP framework is depicted in Algorithm [5J It uses an isolation function /1 to isolate solutions. 
For some integer q, the function /1 maps a solution to a subset of [q]. If and only if two solutions Xi 
and X2 are mapped to subsets with the same cardinality, that is, |/i(a;i)| = |/i(a?2)|) the two solutions 
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compete with each other. In that case, we say the two solutions are in the same isolation, and there 
are at most q + 1 isolations since the subsets of [q] have q + 1 different cardinalities. 
Algorithm 5 (SEIP) 

Given a minimization problem (n,f,C), an isolation function fi encodes each solution as a binary vector of 
length m. SEIP minimization of f with respect to constraint C consists of the following steps. 

1: P^{x® = (0,0,...,0)} 
2: while stop 7^ true do 
3: Choose x £ P uniformly at random 
4: Mutate x to generate x' 
5: if Vcc £ P : superior (a;, a;') = false then 
6: Q <— {x £ P superior(a;', x) = true} 

7: P PU{x'} -Q 

8: end if 
9: end while 
10: return the best feasible solution in P 

where the superior function of two solutions determines whether one solution is superior to the other. This is 
defined as follows: superior(cc, y) is true if both of the following rules are satisfied: 

1) KaO] = Ky)| 

2) f(x) < f(y), or f(x) = f(y) but \x\ < \y\ 
and superior(a;, y) is false otherwise. 



When the isolation function puts all solutions in an isolation for a particular instance, it degrades to 
the (1+1)-EA. The isolation function can also be configured to simulate the dominance relationship of 
multi-objective EAs such as SEMO/GSEMO and DEMO [23]. If we are dealing with fc-objective 
optimization with discrete objective values, a simple approach is to use one of the objective functions, 
say fi , as the fitness function and use the combination of the values of the remaining k — 1 objective 
functions (say $2, ■ ■ ■ , fk) as the isolation functions. Thus, two solutions compete (for /1) only when 
they share the same objective values for /2, ■ • ■ , /fc- This simulation shows that all the non-dominant 
solutions of a multi-objective EA are also kept in the SEIP population, and if a non-dominant solution 
does not reside in the SEIP population, there must be another solution that dominates it. Hence, 
SEIP can be viewed as a generalization of multi-objective EAs in terms of the solutions retained. This 
simulation also reveals that SEIP retains more solutions than a multi-objective EA using the dominance 
relationship. This, on one hand, SEIP takes more time to manipulate a larger population than a multi- 
objective EA does, which could be overcome by defining an isolation function that aggregates nearby 
solutions, as has been done for DEMO 23|. On the other hand, SEIP has more opportunities available 



to find a better approximation solution, since the relationship "a dominates b" does not imply that a 
definitely leads to a better approximation solution than b. 
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Taking the MSCP (n, w, C, U) as an example, we can use the fitness function f(x) = w ■ x, which is 
the sum of costs of the selected sets. For the isolation function, we can use fi(x) as if cc is feasible 
and {1} if cc is infeasible, which isolates the feasible from the infeasible solutions (and thus q = 1); we 
can also use the isolation function fx(x) = UsGxiC)^, and thus the solutions compete only when they 
cover the same number of elements (and thus q — n). 

The mutation operator can use either one-bit or bit-wise mutation. Usually, one-bit mutation is 
considered for local searches, while bit-wise mutation is suitable for global searches as it has a positive 
probability for producing any solution. We denote SEIP with one-bit mutation as LSEIP and SEIP 
with bit-wise mutation as GSEIP, where "L" and "G" denote "local" and "global" , respectively. 

For convenience, SEIP is set to start from solution cc" rather than from a random solution, as commonly 
done for other EAs. Under the condition that any solution will have a better fitness if any 1 bit is 
turned to 0, we can bound the difference between random initialization and starting from cc®. From a 
random solution, SEIP takes at most 0(qm In m) expected steps to find cc according to the following 
argument. Suppose the worst case whereby random initialization generates a solution with all 1 bits; 
according to the fitness function condition, finding cc" is equivalent to solving the OneMax problem 
using a randomized local search and the (1+1)-EA, which takes O(mlnm) steps for both LSEIP and 
GSEIP Q. Furthermore, note that there can be at most q solutions in the population, and thus it 
costs SEIP q expected steps to choose one particular solution from the population. 

The stop criterion is not described in the definition of SEIP, since EAs are usually used as anytime 
algorithms in practice. We now analyze the approximation ratio and the corresponding computational 
complexity of SEIP. 

4. General approximation behavior of SEIP 

For minimization problems, we consider linearly additive isolation functions, (i is a linearly additive 
isolation function if, for some integer q, 

{/i(cc) = [q], for all feasible solutions cc, 

fj,(x U y) = (i(x) U fi(y), for all solutions cc and y. 

The quality of a feasible solution is measured in terms of the approximation ratio. To measure the 
quality of a partial (infeasible) solution, we define a partial reference function and partial ratio as 
follows. 



9 



Definition 4 (Partial reference function) 

Given a set [q] and a value v, a function £[q] jV '■ 2^ — > K is a partial reference function if 

1) £{qlv({q}) = v, 

2) C [g]tV (Ri) = C [q] . v {R 2 ) for all R^R 2 C [q] such that \R t \ = \R 2 \. 

For a minimization problem with optimal cost OPT and an isolation function fi mapping feasible 
solutions to the set [q], we denote a partial reference function with respect to [q] and OPT as C[ q ] opT- 
When the problem and the isolation function are clear, we omit the subscripts and simply denote the 
partial reference function as C. 

Definition 5 (Partial ratio) 

Given a minimization problem (n, /, C) and an isolation function p, the partial ratio of a (partial) 
solution x with respect to a corresponding partial reference function C is 

p-ratio{x) = 

C(p(x)) 

and the conditional partial ratio of y conditioned on x is 

/ I \ f(y\ x) 

p-ratio{x y) = 

£{p(y) | p(x)) 

where f(y | x) = f(x U y) - f(x) and C((i(y) | p(x)) = C(p{y) U p(x)) - C{p{x)). 

The partial ratio is an extension of the approximation ratio. Note that the partial ratio for a feasible 
solution equals its approximation ratio. We have two properties of the partial ratio. One is that it 
is non-increasing in SEIP, as stated in Lemma [TJ and the other is its decomposability, as stated in 
Lemma [21 
Lemma 1 

Given a minimization problem (n, /, C) and an isolation function /i, if SEIP has generated an offspring 
x with partial ratio p with respect to a corresponding partial reference function C, then there is a 
solution y in the population such that \p(y)\ = \p(x)\, and the partial ratio of y is at most p. 

Proof, x is put into the population after it is generated; otherwise there is another solution x' with 
\p(x')\ — \p(x)\ and f(x') < f(x), and in this case let x = x'. The lemma is proved since C(x) = C(y) 
and by the superior function the cost is non- increasing. □ 

From Lemma [T] we know that the partial ratio in each isolation remains non-increasing. Since SEIP 
repeatedly tries to generate solutions in each isolation, SEIP can be considered as optimizing the partial 
ratio in each isolation. 

Lemma 2 

Given a minimization problem (n,f,C) and an isolation function p, for three (partial) solutions x,y 
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and z such that z = xUy, we have 

p-ratio(z) < max{p-ratio(x), p-ratio(y | x)}, 
with respect to a corresponding partial reference function C. 

Proof. Since z = x U y, we have, by definition, 
/(*) = f(x) + f(y | as), 

and 

fi(z) = fi(x U y) = n(x) U n(y). 
Thus, we have 

f M = -^( z ) = f( x ) + f(v \ x ) 
P_rai ° lZj £( M (z)) £(m(x)U M (2/)) 

f(x) + f(y | as) 



£(/*(*)) +£(/i(y) | /*(*)) 



( /(g) f(v I g) 
U(^))' £(a*(») ImW) 



< max ■ 

= max{p-ratio(a;), p-ratio(y | a;)}. 

□ 

Lemma [2] reveals that the partial ratio for a solution is related to the conditional partial ratio of a 
building block. This can be considered as the way in which SEIP optimizes the partial ratio in each 
isolation, that is, by optimizing the conditional partial ratio of each building block partial solution. 
We then have the following theorem. 

Theorem 1 

Given a minimization problem (n, f,C) and an isolation function \i mapping to subsets of [q], assume 
that every solution is encoded in an m-length binary vector. For some constant r > 1 with respect to 
a corresponding partial reference function C, if 

1. p-ratio(x^) < r, 

2. for every partial solution x such that p-ratio(x) < r, SEIP takes x as the parent solution and 
generates an offspring partial solution y such that fi(x) C fi(x U y) and p-ratio(y \ x) < r in 
polynomial time in q and m, 

then SEIP hnds an r-approximate solution in polynomial time in q and m. 
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Proof. Starting from x , we can find a sequence of partial solutions xi,x% . . . ,xt, such that 

x = u| =1 Xj is feasible, 
and that 

Vi = l...t; p-ratio(a; i | x % Up* xj) < r, 

because of the conditions. Note that when a partial solution is added to the solution, the offspring 
solution is in a different isolation to the parent solution. Since the isolation function is linearly additive, 
the length of the sequence I cannot be greater than the number of isolations q + 1 . 

Let t be the time expected for SEIP to generate a partial solution Xi in the sequence from its parent 
solution, which is polynomial to q and m by the condition. It takes at most 0(q) expected steps for 
SEIP to pick the parent, since there are at most q + 1 solutions in the population. Therefore, the total 
time to reach a feasible solution x is 0(t ■ q ■ £), that is, 0(t ■ q 2 ), which is still polynomial in q and m. 

By Lemma [2l since the feasible solution x is composed of x® and partial solutions xi, x% . . . , x m , the 
approximation ratio for x is at most as large as the maximum conditional partial ratio for the partial 
solutions, r. □ 

Theorem [1] reveals how SEIP can work to achieve an approximate solution. Starting from the empty 
set, SEIP uses its mutation operator to generate solutions in all isolations, and finally generates a 
feasible solution. During the process, SEIP repeatedly optimizes the partial ratio in each isolation 
by finding partial solutions with better conditional partial ratios. Since the feasible solution can be 
viewed as a composition of a sequence of partial solutions from the empty set, the approximation ratio 
is related to the conditional partial ratio of each partial solution. 

In Theorem [TJ the approximation ratio is upper-bounded by the maximum conditional partial ratio, 
while some building-block partial solutions may have lower conditional partial ratios but are not uti- 
lized. Moreover, in Theorem Q] we restrict SEIP to append partial solutions, while GSEIP can also 
remove partial solutions using bit-wise mutation. The approximation ratio can have a tighter bound 
if we consider these two issues. Applying the same principle as for Theorem [TJ we present a theorem 
for GSEIP in particular that leads to a tighter approximation ratio. 

Definition 6 (Non- negligible path) 

Given a minimization problem (n, /, C) and an isolation function \i mapping to subsets of [q], assume 
that every solution is encoded in an m-length binary vector. A set of solutions N is a non-negligible 
path with ratios {ri}fZo an< ^ § a P c ^ x® N and, for every solution x g N, there exists a solution 
x' = (x U y + — y~) E N, where the pair of solutions (y + ,y~) satishes 

i. i < \y + \ + \y-\ <c, 
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2. f(y + -y~\x)< r Mx)l ■ OPT, 

3. if\ft(x)\ <q, \ii(xUy+-y-)\ > \fi(x)\. 

Theorem 2 

Given a minimization problem (n, f,C) and an isolation function \i mapping to subsets of [q], assume 
that every solution is encoded in an m-length binary vector. If there exists a non-negligible path 
with ratios {fijfJTQ and gap c, then GSEIP finds an (^?_ ri)-approximate solution in expected time 
0(q 2 m c ). 

Proof. We prove the theorem by tracking the process of GSEIP over the non-negligible path. We 
denote by x cur the solution we want to operate on. Initially, x cur — x®, and thus \fj,(x cur )\ = 0. 

GSEIP takes at most 0(q) expected steps to operate on x cur , since there are at most q + 1 solutions 
in the population and GSEIP selects one to operate on in each step. 

According to the definition of the non- negligible path, there exists a pair of solutions (y + ,y~) with 
respect to x cur such that 1 < \y + \ + \y~\ < c. We denote x 1 = x cur U y + — y . The probability that 
the mutation operator generates solution x' is at least (■=:) c ( Si —^) C ) which implies 0(m c ) expected 
steps. 

According to the definition of the non-negligible path, suppose that \u,(x cur )\ = i; we also have f(y + — 
y | x cur ) < n ■ OPT. Note that f(x') can be decomposed recursively, and thus according to the 
theorem conditions, we have 

/(*') = f(y+ - y | x cur ) + f(x cur ) 
= n -OPT + f(x cur ) = ... 

i i 

= ]T r 3 ■ OPT + f(x*) = ^ fJ • OPT. 

r -OPT 

Let £ be a corresponding partial reference function. Thus, p-ratio(a;') = — 3 c(p( x ')) — ■ 

Given \/j,(x cur )\ — i, again according to the definition of the non-negligible path, we have \/j,(x')\ > 
\fi(x)\. Then we store solution x' in the population; otherwise there exists another solution x" with 
\fi(x")\ = \fj,(x')\ and x" has a smaller partial ratio than x' by Lemma[T]when we substitute x" for x' . 



Now let x cur be x' . We have p-ratio(a; ctir ) < 



r r OPT 



£0(a>')) 

After at most q iterations of the above update of x cur , we have |/i(a; cur )| = q, which means x cur is 

T2 q -Z 1 r OPT E'- 1 r OPT a— 1 

feasible. Thus, the partial ratio of x cur , p-ratio(a; c " , ') = c(^(x^ r )) = — 3 opt = Sj=o r i' ^ s ^ s 

approximation ratio. 
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Thus, at most q jumps are needed to reach a feasible solution, each takes 0{m c ) expected steps for 
operation on a particular solution, and it takes 0(q) expected steps to choose the particular solution. 
Overall, it takes 0{q 2 m c ) expected steps to achieve the feasible solution. □ 

Using Theorem [5] to prove the approximation ratio of GSEIP for a specific problem, we need to find 
a non-negligible path and then calculate the conditional evaluation function for each jump on the 
path. One way of finding a non-negligible path for a problem is to follow an existing algorithm for 
the problem. This will lead to a proof that the EA can achieve the same approximation ratio by 
simulating the existing algorithm. Similar ideas have been used to confirm that EAs can simulate 
dynamic programming 6] . In addition, note that the concrete form of the partial reference function is 
not required in the theorem. 



5. SEIP for the MSCP 



To apply SEIP to the MSCP, we use the fitness function 

f(x) = wx, 

which has the objective of minimizing the total weight. For a solution x, we denote R(x) = U<j eaj (cr)iS, 
that is, R(x) is the set of elements covered by x. We use the isolation function 

H(x) = R(x), 

which, owing to the effect of the isolation function, makes two solutions compete only when they cover 
the same number of elements. We could regard a partial reference function C of x to be the minimum 
price that optimal solutions pay for covering the same number of elements covered by x, although it 
is not necessary to calculate the partial reference function. 

Instead of directly assessing a minimum fc-set cover problem (n,w,C,U,k), we analyze EAs for the 
extended input (n,w',C',U,k) [l^. The original problem is extended by first taking a closure of C 
under the subset operation, that is, C = C U {2 s | MS £ C}, and the weight vector w' is extended 
accordingly by w'(S) — min{ w' (Si ), w'(S2 ),■■•, w '(Sj)} if S C Si, S2, ■ ■ ■ , Sj. Then if an optimal 
solution contains a set with less than fc elements, we construct a new problem instance in which to U 
are added a minimum number of dummy elements such that all sets of the optimal solution are filled 
to be fc-sets using dummy elements while keeping their weights unchanged. Therefore, the extended 
problem has an optimal solution containing fc-sets. Analysis on the extended input leads to the same 
result as for the original problem, as shown in Lemma [3] The lemma is derived from Lemmas 2 and 3 
of Q]. 
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Lemma 3 

The extended input does not affect the optimal cost or the optimal solutions. We can then assume 
without loss of generality that an optimal solution x OPT consists of k-sets {SI, S%, • • • > SI} and is 
disjoint, that is, S* D S* = for all i and j and \S*\ = k for all i. 



Thus, an optimal solution can be represented as a matrix M* of elements, as plotted in Figure[TJ where 
column i corresponds to the elements in S* . Note that there are exactly k rows in M* , since each set in 
an optimal solution contains exactly k elements. For an element e, we denote M*(e) as the column to 
which e belongs, that is, the set in x OPT that contains e. We also denote w*(e) as the cost of M*(e), 
and iV(e) as the number of uncovered elements in column M*(e) at the time at which e is covered. 



S L 



M 



Figure 1: The matrix M* representation 
rows. 



a 



of elements for the minimum fc-set cover problem, containing exactly k 



5.1. SEIP ratio for the (unbounded) MSCP 

In Theorem [3] we show that SEIP achieves an TJ^-approximation ratio, where k is the cardinality of 
the largest set. It has been proved that SEMO [llj| achieves an iJ„-approximation ratio for the MSCP, 
which is known as the asymptotic lower bound for the problem. The theorem confirms that SEIP can 
simulate multi-objective EAs in terms of the solution retained, so that SEIP is able to achieve the 
approximation ratio obtained by multi-objective EAs. 

Theorem 3 

Given an MSCP (n,w,C,U) where \C\ — m, GSEIP Ends an H^-approximate solution in expected 
time 0(mn 2 ), where k is the size of the largest set in C. 

The theorem is proved by simulating the greedy algorithm and using the property of the greedy 
algorithm as in Lemma HI which is derived from [5j. 

Lemma 4 

Given an MSCP (n, w, C, U) and an arbitrary partial solution x, let S — argmin s rg with respect to 
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x. For every element e £ S, there exists a set M*(e) of an optimal solution that covers e, and it holds 
that, 

w(M*(e)) 



price(e) < 



M*(e) - R(x cur )\' 



Proof of Theorem^ We find a non-negligible path following the greedy rule; given the current partial 
solution, add the set S with minimum r^, as in Algorithm[TJ 

We denote x cur as the current solution to be operated on. We find y + = {S} where the set S minimizes 
with respect to x cur . Let y~ = 0. Thus, we have \y + \ + \y~\ = 1 and \fi(x Uy + — y~)\ > |jLt(as)] + 1 
for partial solution x. 

By Lemma SJ for all e G S — R(x cur ), there exists a set M*(e) of an optimal solution that covers e, 
and suppose 

w(M*(e)) 
pmce{ - e > - \M*(e)-R(x^)\- 

In the worst case, \S — R(x cur )\ = 1, that is, the added set only covers one uncovered element e. In 
this case, according to the definition of price(e), we have w(S) = price(e). We then have 

f(y+ - y~ | x cur ) = f(x cw U y+) - f(x cur ) 

= w(S) < ,, f , , * tt • w(M*(e)). 

v ' ~ |M*(e) - R(x cur )\ v v " 

Thus, we find a non-negligible path with gap 1 and sum of ratios 

1 lx ° PTl 1 

^ \M*{e)-R{x^ m -^ M *^= E Y. \ M * {e) _ R{x cur m ^ M *^) 

e j — 1 e £ S j 

\* OPT \ 1 

"EE \M*( e ) - R(x mr (e))\ ' ^ 

l*° PT ] k 1 

~ S W{S;) '^t\M*(e)-R(x^(e))\ 

\n OPT \ 

3=1 

= H k ■ OPT, 

where x cur (e) denotes the partial solution that will cover e in its next step, and k is the size of the 
largest set in C. 

By Theorem [21 GSEIP finds an H^-approximate solution. Note that the isolation function maps to at 
most n isolations, the non- negligible path has a constant gap of 1, and the solution is encoded in an 
m-length binary vector; thus, GSEIP takes expected time 0(mn 2 ). □ 
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Note that in the proof of Theorem[3J with respect to x cur , we find \y \ = and \y + \ = 1. Thus, the 
proof can be adapted to LSEIP directly, as in Theorem SJ 

Theorem 4 

Given an MSCP (n,w,C,U) where \C\ = m, LSEIP finds an iJ^ -approximate solution in expected 
0{mn 2 ) time, where k is the size of the largest set in C. 

5.2. Ratio for GSEIP for the minimum k-set cover problem 

In this section, we prove in Theorem [5] that GSEIP achieves the same (Hk — -|p^-)-approximation ratio 
as GAWW (Algorithm [ljj], the current best algorithm for the minimum fc-set cover problem. This 
result reveals that when the problem is bounded, which is very likely in real-world situations, GSEIP 
can yield a better approximation ratio than in the unbounded situation. Since the greedy algorithm 
cannot achieve an approximation ratio lower than H n , the result also implies that GSEIP has essential 
non-greedy behavior for approximations. 



Given a minimum k-set cover problem (n,w,C,U,k), where \C\ = m, we denote R(x) = Use x S. 
GSEIP using fi(x) = R(x) finds an (Hk — i^r)-approximate solution in expected time 0(m k+1 n 2 ). 

When applying the GAWW rule to select sets, we use Lemmas [5] and El Owing to the assignments of 
price(-), the total price of elements covered by X equals the cost of X. We say a set S is last-covered 
if Ve 6 S : N{e) = 0. 

Lemma 5 (Lemma 4 of [ijj]) 

In each step of GAWW, its partial solution H is disjoint. 
Lemma 6 (Lemma 5 of |13| |) 

In each step of GAWW, we can assume without loss of generality that the set added to X (i.e., S* or 
Q) has no more than one element in common with every set in x . 

Lemmas [7] to \W\ are derived from Lemmas 8, 9 and 10 in jl3l ] and are required for calculating the 
weights of sets that GSEIP selects following the GAWW rule. 



Given a minimum k-set cover problem (n, w, C, U, k) and an arbitrary partial solution x, if the GAWW 
rule uses a withdrawal step to add sets Q — {Si, S2, ■ ■ ■ , Si} to x and withdrawal S 6 x, denoting 
R = UgggS 1 , then 



Theorem 5 



Lemma 7 
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Lemma 8 

Given a minimum k-set cover problem (n, w, G, U, k) and an arbitrary partial solution x, if the GAWW 
rule selects a set S that is not last-covered to add to x (greedy step), and there is an element e'eS 
such that 

priced) <^(e')( I ^ I - i ^), 

then 

, \ w*(e) \ ~* w* (el 
ees y ' ees 

Lemma 9 

Given a minimum k-set cover problem (n, if, G, {/, k) and an arbitrary partial solution x, if the GAWW 
rule selects a set S that is not last-covered to add to x (greedy step), followed by another set S', and 
for all elements e £ S for which 



price(e) > w*(e') 

then 



1 1 



hi + 1 4fc 5 



w*(e) \ ^ w*(e) 



Lemma 10 

Given a minimum k-set cover problem (n, w, G, U, k) and an arbitrary partial solution x, if the GAWW 
rule selects a set S that is last-covered to add to x (greedy step), then 

w*(e) 



<?)<£ eeSiV (e) + l- 

Proof of Theorem^ We find a path of isolations following the GAWW rule. Note that there are at 
most n + 1 isolations. 

For every x cur belonging to an isolation \p(x cur )\ on the path, if the GAWW rule selects sets satisfying 
Lemmas [3 [H |9] and [TUl we find y + containing the sets added and y~ containing the set withdrawn. 
Thus, \y+\ + \y~\ <k + l. 

Since the GAWW rule covers at least one uncovered element, we have \^(x cur Uy + — y~)\ > \fJ,(x cur )\. 

As long as no last-covered set is included, by Lemmas [3 [8] and [9l the partial ratio of f{y + — y 
x cur^ _ w (y+^ _ w(y~) is upper-bounded as 

w*(e) w*(e)" 



w(y + ) - w(y ) < 

ee ( u s ei ,+ M u s e!/ -) 



N(e) + 1 8fc 8 
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otherwise, when the GAWW rule selects a last-covered set using a greedy step, by Lemma ITUI noting 

tr = 0, 



w(y + ) - w(y ) < 



E 



w*(e) 
N(e) + 1 



ee ( u se„+)-( u se,,-) 

In the worst case, there are 1/k among all the elements that are last-covered. Therefore, we find a 
non-negligible path. 



By Theorem [21 GSEIP finds a solution with approximation ratio 

) 



1 /V^fc OPT k - 1 OPT\ k - 1 

Z^ =1 — ; — —)=H k - 



OPT \^i=i i k 
in expected time 0(m k+1 n 2 ). 



□ 



5.3. Comparison of the greedy algorithm, LSEIP and GSEIP 

We assess the greedy algorithm, LSEIP and GSEIP for a subclass of the minimum fc-set cover problem, 
denoted as problem I. We show in Propositions Q] to [3] that LSEIP and GSEIP can overcome the 
difficulty that limit the greedy algorithm, and thus yield better approximation ratios. 

Problem I is a minimum fc-set cover problem (n, w, C, U) constructed as follows. Note that n = kL 
for some integer L. The optimal solution consists of L non-overlapping sets {S*}f =1 , each of which 
contains fc elements in U. Imagine that elements in each S* are ordered and let Sij contain only the 
jth element of set S* . The collection of sets C consists of all sets of S* and Sij . Assign each S* weight 
1 + e for some e > 0, and assign each $y weight l/j. Problem / thus constructed is shown in Figure 

m 



Su 
Sn 







S,,2 





Figure 2: Subclass / of the minimum fc-set cover problem. 



Proposition 1 

Given an arbitrary value £ > 0, the approximation ratio of the greedy algorithm for problem I is 
lower-bounded by Hk — £. 
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Proof. Note that the greedy algorithm adds the set with the minimum cost to the solution at each 
step. On initialization, the cost of S* for all i is i4p, which is higher than the cost i of Sik, the 
smallest cost. Thus, Sik for all i will be added to the solution. Then the cost of S* for all i is vzj, 
which is higher than -j^j for S^k-i)- The greedy algorithm continues to choose a non-optimal set. 
Finally, all sets S 1 ^ will be added to the solution. The approximation ratio is therefore Hk — Hk ■ ■ 
Let £ = Hk ■ j4j. £ can be an arbitrarily small positive value as e can be arbitrarily small. □ 

Proposition 2 

With probability of at least 1 — ^jrj, LSEIP Ends a {Hk — ^(Hk — 1)) -approximate solution for problem 
I in O ((1 + ^)« 3 ) expected steps. 

Proof. Let LSEIP run until there are fc elements uncovered by any solution identified. This takes 
0{mn 2 ) expected steps by Theorem |4j which is 0((1 + k~)n 3 ) since m = (1 + \)n. The uncovered fc 
elements would be covered by sets in {£*}. In the worst case, we assume the k elements are in one set 
St for i. 

Given a solution to be operated on, St is selected with probability ~; set S^ - for j S {1,2,..., k} is 
selected with probability — ; with the remaining probability, no more elements will be covered and we 
return to the solution. Therefore, to cover one of the k elements, 5 1 -* is selected with probability jrjrj-; 
and set for j 6 {1, 2, . . . , k} is selected with probability 

Suppose one set is selected. To cover one more uncovered element, 5 1 -* is selected with probability 
p S- t j for j e {1, 2, . . . , j, . . . , k} is selected with probability ^jp. Therefore, the set S*-* is selected with 
probability 



1 1/ In 1 TT, lx 

fc — + fc (l -m )+ - + 2nd--) 

1 fc+1 1 jfe 12 

+ T ■ 1 7 + ■ 



fc+1 



fc+1 fc+1 fc fc+1 2 fc+1 

fc 1 
_ fc + 1 _ ~ fc + 1' 

and within 0((fc + l)mn) steps, which is overwhelmed by 0(mn 2 ). 

In the worst case, only one set S"~* of an optimal solution is selected for the feasible solution. Thus, the 
approximation ratio is 

(L -'jff. 1 + < < ^H k + i = H k - h Hk - 1) . H k - V* - 1). 
L(l + e) L L L n 

□ 

Proposition 3 

GSEIP finds the optimal solution for problem I in 0(j ■ (1 + j:) k+1 • n k+3 ) expected steps. 
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Proof. Let GSEIP run until a feasible solution is found, which takes 0(mn 2 ) expected steps according 
to Theorem |31 In the worst case, suppose the feasible solution found consists of all sets Sij, so that 
the approximation ratio is Hk — £ for an arbitrary small value £ > 0. 

Keep GSEIP running until an optimal solution is found. GSEIP chooses to operate on the feasible 
solution with probability of at least ^ as there are n isolations, which implies there are 0(n) steps. 

Operating on the feasible solution, for some i, GSEIP uses its mutation operator to replace sets Sij 
for all j and with S* with probability (^^) m ~ k ~ X (h) 1 ■> since k + 1 bits in the solution are to be 
flipped, which implies there are 0(m k+1 ) steps. Once sets Sij for all j are replaced with S*, the partial 
ratio (approximation ratio) decreases and thus the mutated solution is retained. 

Therefore, GSEIP takes 0(mn 2 + nm k+1 L) steps in all; note that 0(mn 2 + nm k+1 L) = 0{mn 2 + 
n 2 m k+1 /k) = 0(\ ■ (1 + i) fc+1 • n k+3 ), since m = n(l + ±). □ 

For GSEIP, we can also derive Proposition HJ proof of which is similar to that for Proposition |3l 
Proposition 4 

GSEIP finds an (Hk — c^(Hk — 1)) -approximate solution for problem I in 0(mn 2 + cn 2 m k+1 /k) ex- 
pected steps, for c = 0, 1, . . . , 

This proposition illustrates an interesting property: compared with the greedy algorithm, whose per- 
formance cannot be improved given extra time, SEIP always seeks better approximate solutions. Users 
can allocate more running time in a trade-off for better solutions. However, the solution quality may 
not be improved by an EA over a long time for practical HP-hard problems. Thus, users should not 
assume any useful relationship between time allocated and the approximate ratio. 



6. Conclusion 

We studied the approximation performance of an EA framework. SEIP introduces an isolation function 
to manage competition among solutions, which can be configured as a single- or multi-objective EA. 
We analyzed the approximation performance of SEIP using the partial ratio and obtained a general 
characterization of the SEIP approximation behavior. Our analysis confirms that SEIP can achieve a 
guaranteed approximate solution: it tries to optimize the partial ratio of solutions in every isolation 
by finding good partial solutions, then these partial solutions form a feasible solution with a good 
approximation ratio. 

We studied the performance of SEIP for the MSCP, which is NP-hard. Previous studies [ll|, Q 
showed that (1+1)-EA is not a good solver for MSCP, while a multi-objective EA achieves a similar 
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approximation ratio to that of the greedy algorithm. Our analysis extends previous work to show 
that for the unbounded MSCP, SEIP achieves an iffe-approximation ratio (where Hk is the harmonic 
number of the cardinality of the largest set), the asymptotic lower bound. For the minimum fc-set 
cover problem, it achieves an (H^ — ^^-approximation ratio, the current best-achievable result 
which is beyond the ability of the greedy algorithm. Moreover, for a subclass of the MSCP, we show 
how SEIP with either one-bit or bit-wise mutation can overcome the difficulty that limits the greedy 
algorithm. 

We discussed some advantages and limitations of SEIP for approximations. To prove the SEIP ap- 
proximation ratio for the MCSP problem, we used SEIP to simulate the greedy algorithm. Since the 
greedy algorithm is a general scheme for approximations and has been analyzed for many problems, 
SEIP analysis can be extended to cases for which the greedy algorithm has been applied, and therefore 
we can easily show that SEIP is a ^-approximation algorithm for fc-extensible systems [21], including 
^-matching, maximum profit scheduling and maximum asymmetric TSP problems. Moreover, to prove 
the approximation ratio of SEIP for the minimum fc-set cover problem, we used SEIP to simulate the 
GAWW algorithm, which implies that SEIP also has extra behaviors that provide opportunities to 
exceed the greedy algorithm. However, limitations of SEIP are found from Theorem [5] There are 
some situations in which SEIP may fail. SEIP is required to flip a number of bits at a time, which 
depends on n, to achieve a good solution. In this situation, to flip the required number of bits, one-bit 
mutation is limited since it only flips one bit at a time, and bit-wise mutation is also limited as it 
requires exponential time. Further designs for elegant mutation operators may be possible. However, 
since the solution space has size 2™, any mutation has to assign an exponentially small probability 
to some distant mutations. Once these distant mutations are required, exponential time is required. 
Another view of this limitation is that since the mutation operator only allows limited steps, SEIP 
may also over-optimize partial ratios, just like the greedy algorithm over-optimizes each step. 

Our theoretical analysis suggests that EAs can achieve solutions with guaranteed performance. We 
believe that guaranteed and better approximation performance can be achieved by better EA design 
in the future. 
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